1. Technical Field
This disclosure relates to iterative estimates of an unknown parameter of a model or state of a system.
2. Description of Related Art
The expectation-maximization (EM) algorithm is an iterative statistical algorithm that estimates maximum-likelihood parameters from incomplete or corrupted data. See A. P. Dempster, N. M. Laird and D. B. Rubin, “Maximum Likelihood from Incomplete Data via the EM Algorithm (with discussion),” Journal of the Royal Statistical Society, Series B 39 (1977) 1-38; G. J. McLachlan and T. Krishnan, The EM Algorithm and Extensions (John Wiley and Sons, 2007); M. R. Gupta and Y. Chen, “Theory and Use of the EM Algorithm,” Foundations and Trends in Signal Processing 4 (2010) 223-296. This algorithm has a wide array of applications that include data clustering, see G. Celeux and G. Govaert, “A Classification EM Algorithm for Clustering and Two Stochastic Versions,” Computational Statistics & Data Analysis 14 (1992) 315-332; C. Ambroise, M. Dang and G. Govaert, “Clustering of spatial data by the em algorithm,” Quantitative Geology and Geostatistics 9 (1997) 493-504, automated speech recognition, see L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proceedings of the IEEE 77 (1989) 257-286; B. H. Juang and L. R. Rabiner, “Hidden Markov models for speech recognition,” Technometrics 33 (1991) 251-272, medical imaging, see L. A. Shepp and Y. Vardi, “Maximum likelihood reconstruction for emission tomography,” IEEE Transactions on Medical Imaging 1 (1982) 113-122; Y. Zhang, M. Brady and S. Smith, “Segmentation of Brain MR Images through a Hidden Markov Random Field Model and the Expectation-Maximization Algorithm,” IEEE Transactions on Medical Imaging 20 (2001) 45-57, genome-sequencing, see C. E. Lawrence and A. A. Reilly, “An expectation maximization (EM) algorithm for the identification and characterization of common sites in unaligned biopolymer sequences,” Proteins: Structure, Function, and Bioinformatics 7 (1990) 41-51; T. L. Bailey and C. Elkan, “Unsupervised learning of multiple motifs in biopolymers using expectation maximization,” Machine learning 21 (1995) 51-80, radar denoising, see J. Wang, A. Dogandzic and A. Nehorai, “Maximum likelihood estimation of compound-gaussian clutter and target parameters,” IEEE Transactions on Signal Processing 54 (2006) 3884-3898, and infectious-disease tracking, see M. Reilly and E. Lawlor, “A likelihood-based method of identifying contaminated lots of blood product,” International Journal of Epidemiology 28 (1999) 787-792; P. Bacchetti, “Estimating the incubation period of AIDS by comparing population infection and diagnosis patterns,” Journal of the American Statistical Association 85 (1990) 1002-1008. A prominent mathematical modeler has even said that the EM algorithm is “as close as data analysis algorithms come to a free lunch”, see N. A. Gershenfeld, The Nature of Mathematical Modeling (Cambridge University Press, 1999). But the EM algorithm can converge slowly for high-dimensional parameter spaces or when the algorithm needs to estimate large amounts of missing information, see G. J. McLachlan and T. Krishnan, The EM Algorithm and Extensions (John Wiley and Sons, 2007); M. A. Tanner, Tools for Statistical Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions, Springer Series in Statistics (Springer, 1996).